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PROBLEM TO BE SOLVED: To simply convert an actually photographed picture into a handwritten-like 
picture at low cost by omitting a conventional handwritten picture preparation process requiring a person 
and a time. 

SOLUTION: A picture processor for generating a handwritten-like video based on an actually 
photographed video is provided with a frame memory 2 for input for an input picture, picture processor 3 
for picture processing the picture signal of the frame memory 2 for input, parameter inputting device 4 for 
inputting a parameter for the picture processing of the picture processor 3, and frame memory 5 for 
output for storing the picture processed result from the picture processor 3. At that time, a picture signal 
inputted to the picture processor 3 is processed in an edge detecting filter so that edge strength and the 
inclination can be detected, and a touch shape can be decided. 
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CLAIMS 



(57) [Claim(s)] 

[Claim 1] The image processing system which generates the image of the hand-drawn style based on the real map 
image which is equipped with the following, filters [ edge-detection ] the picture signal inputted into the 
aforementioned picture processor, detects edge intensity and its inclination, and is characterized by determining a touch 
configuration. The frame memory for an input for input pictures. The picture processor which carries out the image 
processing of the picture signal of this frame memory for an input. Parameter input equipment which inputs a 
parameter for the image processing of this picture processor. The frame memory for an output which stores the image- 
processing result by the aforementioned picture processor. 

[Claim 2] The image processing system characterized by repeating edge-detection filtering the picture signal inputted 
by the aforementioned picture processor in an image processing system according to claim 1 , detecting edge intensity 
and its inclination, and determining a touch configuration the number of predetermined times. 

[Claim 3] The image processing system which carries out image transformation based on the real map image which is 
equipped with the following, filters [ edge-detection ] the picture signal inputted into the aforementioned picture 
processor, sets up the size and the coefficient of this edge-detection filter, detects the writing pressure parameter as one 
of the key touch attributes, detects the aforementioned edge intensity and its inclination, and is characterized by 
repeating determining a touch configuration the number of predetermined times. The frame memory for an input for 
input pictures. The picture processor which carries out the image processing of the picture signal of this frame memory 
for an input. Parameter input equipment which inputs a parameter for the image processing of this picture processor. 
The frame memory for an output which stores the image-processing result by the aforementioned picture processor. 
[Claim 4] The image processing system characterized by repeating edge-detection filtering the picture signal inputted 
into the aforementioned picture processor in an image processing system according to claim 3, detecting edge intensity 
and its inclination, attaching a shade to a color by the strength of an edge, and determining a touch configuration the 
number of predetermined times. 

[Claim 5] The image processing system characterized by repeating edge-detection filtering the picture signal inputted 
by the aforementioned picture processor in an image processing system according to claim 3, detecting edge intensity 
and its inclination, and not performing hand-drawn depiction for the aforementioned edge intensity in below a 
threshold, and determining a touch configuration the number of predetermined times. 

[Claim 6] A picture signal is inputted in the image-processing method which carries out the image processing of the 
picture signal. Set up the parameter of the length size of a touch, and the aforementioned touch draws and the number 
of times of a lump is set up. (a) An edge-detection filter is covered over the coordinate P in a frame (x y) at level and a 
perpendicular direction, (b) Incline with the edge intensity s (sx, sy) of the aforementioned coordinate P (x y), and it 
asks for theta. (c) The image-processing method characterized by inclining with this edge intensity s (sx, sy), 
determining a touch configuration from theta and the aforementioned parameter, and for the (d) aforementioned touch 
drawing, and performing the step of (c) several setup from the above (a) according to the number of times of a lump. 
[Claim 7] The image-processing method which sets up an edge on-the-strength threshold further, and is characterized 
by determining the touch configuration by the above (c) when larger than the edge intensity s (sx, sy) and the 
aforementioned edge on-the-strength threshold by the above (b) in the image-processing method according to claim 6. 
[Claim 8] The image-processing method characterized by performing execution of (d), storing in an output frame 
memory, reading from this output frame memory, and considering as an resolution picture after inputting the 
aforementioned picture signal into a frame memory and determining the touch configuration of the above (c) in the 
image-processing method according to claim 6 or 7. 

[Claim 9] The aforementioned picture signal is the claim 6 characterized by being a picture signal of the hand-drawn 
style based on [ are the picture signal of an on-the-spot photo, and ] the picture signal of the aforementioned on-the- 



spot photo in the aforementioned resolution picture, or the image-processing method given in 7 and 8. 



[Translation done.] 
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l.This document has been translated by computer. So the translation may not reflect the original precisely. 
2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[The technical field to which invention belongs] Especially this invention relates to the image-processing method 
which generates the image of the hand-drawn style based on a real map image, and its equipment about an image 
processing system. 
[0002] 

Background of the Invention] In recent years, digitization of a picture signal is made known frequently and the special 
image expression of it which compounded the real map image and the computer graphics image in digital one is 
attained. Main positions are occupied in a graphic commercial scene. However, in computer graphics, there is an image 
which gives an impression which human being drew by hand as what has difficult generation. Although a straight line 
and an inclination line can be easily described at computer graphics, in order to make analog-depiction poor, it is not 
even easy to draw a perfect circle. Although it uses a dither method, the carrying-out-smoothly technique, etc. of 
having taken into consideration especially the smoothness of the same character representation as the printer which 
indicates by the dot in digital one, it is difficult to model very like hand-drawn. Moreover, there are not many image 
processing systems of the hand-drawn style which do not depend on the interactive operation between user equipment 
and which can be processed automatically. 
[0003] 

[Problem(s) to be Solved by the Invention] Thus, the conventional hand-drawn drawing creation process (even when 
creating an image by interactive operation, time ** for mastering people and the operating instruction which have **** 
to some extent after all is needed) of requiring people and time is omitted, and as easy as possible and the realization of 
a means (equipment) which is a low cost and generates the image of the hand-drawn style are required technical 
problems technically. 

[0004] this invention aims at changing into the picture of the hand-drawn picture style from an on-the-spot photo 

picture especially for the purpose of solving the above-mentioned technical problem. 

[0005] 

[Means for Solving the Problem] this invention consists of parameter input equipment for the equipment for 
accumulating an input image, the frame memory for input frames, the frame memory for output frames, the picture 
processor for actually processing an input frame in the picture of the hand-drawn style, and a user controlling a picture 
processor. 

[0006] In the image processing of this invention, with the edge-detection filter which can be processed high-speed, a 
border line is detected and a touch is written in in the detected direction of a border line. A picture processor has an 
image-processing function based on this edge detection. Moreover, the writing of the touch in this invention means 
writing fields, such as a rectangle or an ellipse, in an output frame memory. Moreover, the color in the field written in 
is decided based on the sexual desire news of a subject-copy image. Here, the reason for writing in a touch in the 
direction of a border line is based on assumption of drawing without breaking down a profile configuration, when 
human being draws a picture. 

[0007] Realization of "the hand-drawn wind" in this invention is based on the following ideas. The pictures by hand- 
drawn [ in the real world ] are created by drawing and repeating the touch on the canvas which is **** for oil painting. 
In case the profile portion of a drawing object object is drawn, we make a brush run along the direction of the profile 
(it is thought that this is because "direction detection" of a body profile is performed without our being conscious and 
the brush is made to run according to the direction). 

[0008] Moreover, it is thought that the "touch" which is a hand has attributes, such as a configuration, length, a size, a 
direction, and a color. If the procedure and "a touch" of drawing in such the real world are modeled, it will become 
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possible to perform automatic processing. ** That is, detect the direction of an objective profile portion from the 
existing picture, by what (a parameter expresses a touch) ** touch is modeled for, copy the procedure at the time of our 
actually drawing, and, as for this invention, realize "a hand-drawn wind." 

[0009] In the image processing system with which this invention specifically generates the image of the hand-drawn 
style based on a real map image The frame memory for an input for input pictures, and the picture processor which 
carries out the image processing of the picture signal of this frame memory for an input, The parameter input 
equipment which inputs a parameter for the image processing of this picture processor, It consists of a frame memory 
for an output which stores the image-processing result by the aforementioned picture processor, the picture signal 
inputted by the aforementioned picture processor is edge-detection filtered, edge intensity and its inclination are 
detected, and it is characterized by determining a touch configuration. 

[0010] Moreover, it sets to the image processing system which generates the image of the hand-drawn style based on a 
real map image. The frame memory for an input for input pictures, and the picture processor which carries out the 
image processing of the picture signal of this frame memory for an input, The parameter input equipment which inputs 
a parameter for the image processing of this picture processor, The frame memory for an output which stores the 
image-processing result by the aforementioned picture processor is provided. The picture signal inputted by the 
aforementioned picture processor is edge-detection filtered. The size and the coefficient of this edge-detection filter are 
set up, the writing pressure parameter as one of the key touch attributes is introduced, edge intensity and its inclination 
are detected, and it is characterized by repeating determining a touch configuration the number of predetermined times. 

[0011] 

[Embodiments of the Invention] 

The 1st operation form by the [operation form of ** 1st] this invention is explained in detail, referring to a drawing. 
The whole image processing system composition of this operation form is shown in drawing 1 . In drawing 1 , 1 is 
input image accumulation equipment which changes real map picture signals, such as a photograph and a video image, 
into a digital signal by the non-illustrated A/D converter, and memorizes them. The image data which 2 is a frame 
memory for accumulating the digital image for one frame which was read from input image accumulation equipment 1 
every one frame, and was inputted, and was accumulated at this memory is processed by the picture processor of 3, and 
a processing picture is written in the frame memory of 5. Moreover, 4 is equipment used for control of the picture 
processor 3, and a parameter setup. Moreover, the data in the frame memory of 5 are accumulated by making a picture 
frame into a continuous picture signal at the output image accumulation equipment of 6. 

[0012] In addition, as mentioned above, although the A/D converter is arranged between a real map picture signal and 
input image accumulation equipment 1 and used input image accumulation equipment 1 as a digital signal storage 
means, it is arranged between input image accumulation equipment 1 and a frame memory 2, and is good also 
considering input image accumulation equipment 1 as an analog storage means. Moreover, similarly, although it is 
easy to be as above-mentioned when output image accumulation equipment 6 is a digital formula storage means, in the 
case of an analog formula storage means, the output of the picture processor 3 may be changed into an analog video 
signal by the D/A converter, and you may store in output image accumulation equipment 6. 

[0013] Moreover, although frame memories 2 and 5 are used as one frame memory, they become possible [ presenting 
an image processing colorful also as two or more frame memories with at least one field memory ]. For example, when 
the picture which picturized especially two photographs is stored in each frame memory in the case of two or more 
frame memories, a new hand-drawn picture signal can be created for the hand-drawn picture in one frame, and the 
following hand-drawn picture combining the average of for example, each coordinate, and a creative picture can be 
generated. 

[0014] Next, the processing in the picture processor of 3 shown in drawing 1 is explained with reference to the flow 
chart shown in drawing 2 . 

[0015] In drawing 2 , two or more parameters for hand-drawn are first set up from the input picture accumulated first at 
the frame memory 2 of drawing 1 (SI). Next, the number of times of writing of a touch and the variable loop for the 
number of times are initialized (S2). Next, the coordinate P of arbitrary coordinate points (x y) is chosen at random 
(S3), x shows a position horizontal as a reference point for the screen upper left, and y shows a vertical position here. 
Two edge-detection filters of level and a perpendicular direction are applied to the coordinate value P (x y), and a value 
Q (sx, sy) is acquired (S4). 

[0016] Here, a "edge-detection filter" is a filter of for example, 3x3 matrix, and a perpendicular direction detection 
filter and a horizontal detection filter are used for it. It asks for the matrix product of the brightness value of each pixel 
inside the 3x3 field centering on a certain coordinate (point shown by O), and the filter (perpendicular in it being level) 
of 3x3 matrices, and the value over each horizontal direction and perpendicular direction is acquired. It is theta=arctan 
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(Y/X) about the direction (theta) of an edge in the central point (O) from the group (X, Y) of the acquired value. 
^ can come out and ask. 

[0017] Moreover, what can be processed high-speed is because it is calculable to a filter factor by addition of the 
brightness value which is each pixel, and subtraction since there are much 1 and -1 . 
[0018] For example, the following 3x3 matrix picture is explained. 
[0019] 

x x xx O xx x x — in this picture, a thing typical as an example of an edge-detection filter is shown below The operator 
of Prewitt (perpendicular) 

- 1 0 1-1 0 1-1 0 Operator of 1 Prewitt (Level) 

-1 -1 -1 

0 0 0 
-1 1 1 

The operator of Sobel (perpendicular) 

- 1 0 1-2 0 2-1 0 Operator of 1 Sobel (Level) 

-1 -2 -1 

0 0 0 
-1 2 1 

However, when an input image is a color, a filter is applied to a brightness value. 

[0020] Next, edge intensity in the coordinate value P (x y) from obtained Q (sx, sy), s=sqrt (sx*sx+sy*sy) .... (1) 
Inclination, theta=arctan (sy/sx) .... (2) 

Eye **** (S5), the parameter set up by SI, intensity s, and the touch which determines a touch configuration from 
theta by inclining, and has the angle of Inclination theta by the color (R, G, B) of Coordinate P (x y) are written in a 
frame memory 5 (S6). Next, the variable loop for the number of times is added one time (S7), and it judges whether the 
variable loop for the number of times is larger than the set point (S8), and when small, it shifts to S3. In this way, when 
only the number of times of specification repeats this loop, the image processing of the hand-drawn style is performed. 
[0021] Thus, the picture of the handwriting style can be described by inclining with the intensity s of an edge and 
giving the length and the size of a touch to theta (inclination of coordinate P obtained from the product of a 
circumference pixel and a filter focusing on P (x y)), and a parameter, moreover, a touch shows small fields, such as a 
rectangle and an ellipse, here, and a "smallness field" is seen and built in the configuration of an actual hand (for 
example, what is necessary is just to make the configuration of a "smallness field" into a perfect circle, in order to copy 
a hand like a sketch) In addition, the color in Coordinate P is set as the color in a field. The condition judging of 
drawing 2 of S8 changes Coordinate P (x y), and when loop count has not reached a closing price, it changes. In this 
way, if the "smallness field" (= touch) is repeated and written in, since the coordinate is chosen at random and an 
output frame will be filled with a touch almost uniformly, it becomes the style of handwriting. In order that the field to 
write in may write in along the fully small thing and the direction which detected the field compared with picture size, 
even if it writes at random and piles up, a result does not go to pieces. 

[0022] About a coordinate selection criterion and a drawing procedure, a coordinate is chosen at random and there is 
the way of writing the touch in order with the small edge intensity of the way (the output frame memory 5 is 
overwritten) of overwriting the touch and (1) (2) coordinate. 

[0023] Under the present circumstances, it asks for intensity s using a formula (1). In the case of a procedure (2), the 
procedure written in "order with small edge intensity" is shown below. All points with the on-the-strength value s equal 
to a certain value n are discovered, and a touch is written in the position. If the value of n is changed from the 
minimum value of the on-the-strength value s to maximum, a touch can be written in order with small edge intensity. 
[0024] In addition, naturally a flow chart changes in a procedure (2) and a procedure (1). Moreover, since a coordinate 
(place where the clear edge exists) with large edge intensity is drawn at the end, the advantage of the method of (2) has 
the preservation effect of the edge of a subject-copy image larger than mere random writing, moreover, way touch 
writing overwrites an input picture or the output frame memory 5 — for example, the method of overwriting, after 
carrying out initial setting white etc. — ****** . 

[0025] The result which made the subject-copy image drawing 5 and carried out the image processing of the example 
of drawing by this operation gestalt in the style of handwriting to the subject-copy image is shown in drawin g 6 . The 
size of the edge on-the-strength value s means the height of possibility that an edge exists. Moreover, if the touch is 
written in the coordinate with high possibility of being an edge, the direction of the edge in the coordinate will not shift 
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from it of a subject-copy image greatly. That is, it will be said that possibility that the direction of an edge in a subject- 
£opy image is saved surely is high. 

[0026] Moreover, the frame memory considers that it is the two-dimensional array of a subject-copy image and the 
same size, and touch writing is the memory holding each value of (R, G, B), and it can be performed by writing the 
color (each value of R, G, and B) of Coordinate P in the coordinate included in "a field (= touch)" centering on the 
position (x y) of the coordinate P chosen by S3 of drawing 2 at two-dimensional array. 

[0027] Furthermore, since it is common that the thing with white actual canvas and actual drawing paper carries out 
initial setting white, the initial value of the two-dimensional array holding an output picture is set up white. By carrying 
out like this, it is thought that it becomes possible to copy process in which paints are actually recoated with pen in 
canvas or drawing paper. That is, they are "the canvas and drawing paper = two-dimensional array" in the real world. 
[0028] The 2nd operation form of a [operation form of** 2nd] this invention is explained referring to a drawing. The 
step which performs smoothing on an input image as pretreatment between a start step and each parameter setting step 
(SI) is added, and it enables it to direct from the parameter input equipment 7 shown in drawing 3 in the processing 
flow shown in drawing 2 . Parameter input equipment 7 has each parameter input section 8, the on-the-strength sensor 
section 9, and a ten key 10, and is arranged between the input image accumulation equipment 1 of drawing. 1 , and a 
frame memory 2. 

[0029] Moreover, parameter input equipment 7 is equipped with a ten key 10, and it enables it to set up the size and the 
coefficient of a filter about the edge-detection filter to be used. A writing pressure parameter is introduced as one of the 
key touch attributes, and in order to perform this parameter control, parameter input equipment is equipped with the 
input devices (a touch panel, button, etc.) which have an on-the-strength sensor function ( drawing.3 ). By actually 
tracing a panel using a pen etc., a user can tell favorite writing pressure to an image processing system. 
[0030] Here, since 9 of drawing J> is an on-the-strength sensor, tracing inputs strength by pushing this front face. A 
sensor (9 of drawing 3 ) detects "writing pressure", and it is substituted for the variable of the "writing pressure 
parameter" in an image processing processor for the value corresponding to the pushed strength. This parameter is one 
of the attributes about a touch. 

[0031] In addition, as other attributes about a touch, there are the configurations (a rectangle, an ellipse, perfect circle, 
etc.) of (1) touch, the length of (2) touches, (3) sizes, a color of (4) touches, and a direction that writes in (5) touches. 
Among these, (1) touch configuration, (2) length, and (3) sizes are set up by SI of drawing 2 , and (4) colors and the 
(5) directions are determined in process of S6 of drawing 2 . (4) The color of a touch can set up the color of the 
selected coordinate P, and it can ask for the (5) directions from a formula (2). 

[0032] According to this method, by the writing pressure specification by the numeric value, the fault of being hard to 
take correspondence with actual writing pressure feeling is avoidable. 

[0033] The edge on-the-strength value acquired by application of an edge-detection filter besides only using the color 
in a selection coordinate is used for the determination of the color of [determination of color of key touch] key touch. 
This method has the feature which has a shade attached to a color by the strength of an edge. This procedure is shown 
drawing 4. 

[0034] In drawing^ , the processing before application of an edge-detection filter and the processing after touch 
writing are the same as the flow of drawing^ . Now, the intensity s of a touch can be found by filter application. First, 
value s" which normalized s to the real numeric values from 0 to 1 is calculated. 
[0035] 

s"<- s/ MAX_VAL .... (3) 

MAX_VAL which appears in **S12 of drawing 4 means the maximum on-the-strength value of edge intensity. The 
maximum on-the-strength value of edge intensity is the maximum on-the-strength value in [ all / in one frame ] a pixel. 
Therefore, although the step which calculates maximum from the inside of one frame is required, it is omitting on 
drawing. By the formula of **S12 of drawing 4 , the on-the-strength value of the all pixel in one frame is normalized 
to[0..1]. 

[0036] Next, the color of a touch is calculated using the following color formula which made s" weight (**S13 of 

drawing 4 ). 

[0037] 

r <- s"xR+(l-s") xMAX_Y .... (4) 
g <- s"xG+(l-s") xMAX.Y .... (5) 
b <- s"xB+(l-s") xMAX_Y .... (6) 

Here, R, G, and B are the colors of the subject-copy image in the selected coordinate P, and MAX_Y expresses the 
maximum brightness value (R, G, B if it is 8-bit data respectively MAX_Y= 255). if it is s"=0 (there is no edge) by this 
color - a touch - white and s" - if it becomes =1 (there is an edge), a touch will become the color of Coordinate P 
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itself, and it becomes possible to attach a shade to a color by the size (strength of an edge) of s" Draw ing 8 is the 
example of the picture corresponding to drawing 7 of the on-the-spot photo subject copy obtained by this method. In 
this case, it turns out that atmosphere like a watercolor painting is expressed. 

[0038] The margin of canvas is in one of the features of the picture appropriate for [control of margin of canvas] hand- 
drawn. Canvas is not completely covered by touch, but when a margin is partially transparent and it is visible, it 
becomes hand-drawn ****** sensibility more. Margin control of canvas is performed on being based on the edge 
intensity calculated with the edge-detection filter. A threshold is set up, and when an edge on-the-strength value is 
smaller than this, it is made not to write in a touch as an art. It is the point that a margin remains in a portion (portion 
with loose = brightness change) with small edge intensity as an advantage of this method, and an important profile 
portion does not become blank visually. Furthermore, it is also possible by changing a threshold to write a touch only 
in a profile portion. 

[0039] The flow chart in this case is shown in drawin g 9 . From the input picture accumulated first at the frame 
memory 2 of drawing 1 , two or more parameters for hand-drawn are set up first (S21). Next, the on-the-strength 
threshold of an edge is set up (S22). Since the difference in in the hand-drawn and the subject copy which serve as the 
feature by the edge detection when edge intensity is low is small, it is for leaving the state of a subject copy as it is. 
[0040] Next, the number of times of writing of a touch and the becoming variable loop for the number of times are 
initialized (S23). Next, the coordinate P of arbitrary coordinate points (x y) is chosen at random (S24). Next, an edge- 
detection filter is covered over level and a perpendicular direction with Coordinate P (x y), and it asks for the group S 
of the value of the edge intensity (sx, sy) (S25). This edge intensity may be the same as that of the edge-detection filter 
which was explained in the case of drawing 2 , or may ask for edge intensity by the edge detection using the linear 
differential operator and alternative partial equalization by Laplacian, the low-pass emphasis filter, and the high region 
emphasis filter. 

[0041] Next, the edge intensity s in the coordinate value P (x y) from obtained S (sx, sy), s=sqrt (sx*sx+sy*sy) .... (3) 
Inclination theta, theta=arctan (sy/sx) .... (4) 

Eye **** (S26) and the edge intensity s compare with the edge on-the-strength threshold set up at Step S22 (S27). 
When smaller than an edge on-the-strength threshold, it shifts to Step S24, and in being larger than a threshold, from 
the set-up parameter, intensity s, and inclination theta, the edge intensity s determines a touch configuration and writes 
the touch configuration of Coordinate P (x y) in a frame memory 5 (S28). Next, the variable loop for the number of 
times is added one time (S29), and it judges whether the variable loop for the number of times is larger than the set 
point (S30), and when small, it shifts to S24. In this way, when only the number of times of specification repeats this 
loop, the picture of the hand-drawn style can be acquired by storing in the output frame memory 5 the touch 
configuration of the coordinate P (x y) chosen at random, and acquiring the picture signal of one frame from the output 
frame memory 5. With this operation form, by setting an on-the-strength threshold as a larger value, it becomes 
possible to draw only a profile portion and it can express the picture of the hand-drawn style. 
[0042] 

[Explanation of an effect] According to this invention, "conversion of an on-the-spot photo image to the image of the 
hand-drawn style" different from "conversion of an on-the-spot photo image to a real map image" of rotation, resizing, 
change and subtractive color of a color, composition, etc. becomes possible. That is, the new image processing which 
is not due to a real map image and CG image becomes possible, and colorful picture depiction can be expressed. 
[0043] Moreover, conventionally, it is the technology used as the alternative means of the hand-drawn drawing 
creation depending on human being, and a maker's art can be expressed effectively. 

[0044] Moreover, it is the technology which enables reuse in the form where the past image property was different. 
That is, it is possible to modify art owner ********** an d pictures of a past many, and the further art can be 
demonstrated easily. 
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